夜间场景解析(NTSP)对于许多视觉应用是必不可少的,尤其是对于自动驾驶。大多数现有方法都是为了解析白天的现有方法。他们依靠在照明下建模基于像素强度的空间上下文线索。因此,这些方法在夜间场景中表现不佳,因为这种空间上下文提示被埋葬在夜间场景中的过度/暴露区域中。在本文中,我们首先进行了基于图像频率的统计实验来解释白天和夜间场景差异。我们发现,在白天和夜间场景之间,图像频率分布有很大差异,并且了解此类频率分布对于NTSP问题至关重要。基于此,我们建议利用图像频率分布来解析夜间场景。首先,我们提出了一个可学习的频率编码器(LFE),以模拟不同频率系数之间的关系,以动态测量所有频率组件。其次,我们提出了一个空间频率融合模块(SFF),该模块融合了空间和频率信息,以指导空间上下文特征的提取。广泛的实验表明,我们的方法对夜总会,夜城+和BDD100K晚数据集的最先进方法表现出色。此外,我们证明我们的方法可以应用于现有的白天场景解析方法,并在夜间场景中提高其性能。
translated by 谷歌翻译
时空预测学习是通过历史先验知识来预测未来的框架变化。以前的工作通过使网络更广泛和更深入来改善性能,但这也带来了巨大的内存开销,这严重阻碍了技术的开发和应用。比例是提高普通计算机视觉任务中模型性能的另一个维度,这可以减少计算要求并更好地感知环境。最近的RNN模型尚未考虑和探索如此重要的维度。在本文中,我们从多尺度的好处中学习,我们提出了一个名为多尺度RNN(MS-RNN)的通用框架,以增强最近的RNN模型。我们通过在4个不同的数据集上使用6种流行的RNN模型(Convlstm,Trajgru,Predrnn,Prodrnn ++,MIM和MotionRNN)进行详尽的实验来验证MS-RNN框架。结果表明,将RNN模型纳入我们的框架的效率低得多,但性能比以前更好。我们的代码在\ url {https://github.com/mazhf/ms-rnn}上发布。
translated by 谷歌翻译
我们总结了使用巨大的自动语音识别(ASR)模型的大量努力的结果,该模型使用包含大约一百万小时音频的大型,多样的未标记数据集进行了预训练。我们发现,即使对于拥有数万个小时的标记数据的非常大的任务,预训练,自我培训和扩大模型大小的组合也大大提高了数据效率。特别是,在具有34K小时标记数据的ASR任务上,通过微调80亿个参数预先训练的构象异构体模型,我们可以匹配最先进的(SOTA)性能(SOTA)的性能,只有3%的培训数据和通过完整的训练集可以显着改善SOTA。我们还报告了从使用大型预训练和自我训练的模型来完成一系列下游任务所获得的普遍利益,这些任务涵盖了广泛的语音域,并涵盖了多个数据集大小的大小,包括在许多人中获得SOTA性能公共基准。此外,我们利用预先训练的网络的学会表示,在非ASR任务上实现SOTA结果。
translated by 谷歌翻译
复杂网络包含完整的子图,例如节点,边缘,三角形等,称为不同订单的简单和批分。值得注意的是,由高阶派系组成的空腔在大脑功能中起重要作用。由于搜索最大批变是一个NP完整的问题,因此我们使用K-Core分解来确定给定网络的可计算性。对于可计算的网络,我们设计具有可实现的算法的搜索方法,用于查找不同订单的Cliques,还获得欧拉特征数。然后,我们通过使用相邻派系的边界矩阵的级别来计算Betti号。此外,我们设计了一种用于查找不同订单的空腔的优化算法。最后,我们将该算法应用于来自一个典型数据集的数据的C.杆杆线虫的神经元网络,并找到其所有不同订单的群体和一些空腔,为其结构和功能提供了进一步的数学分析和计算的基础。
translated by 谷歌翻译
Along with the springing up of semantics-empowered communication (SemCom) researches, it is now witnessing an unprecedentedly growing interest towards a wide range of aspects (e.g., theories, applications, metrics and implementations) in both academia and industry. In this work, we primarily aim to provide a comprehensive survey on both the background and research taxonomy, as well as a detailed technical tutorial. Specifically, we start by reviewing the literature and answering the "what" and "why" questions in semantic transmissions. Afterwards, we present corresponding ecosystems, including theories, metrics, datasets and toolkits, on top of which the taxonomy for research directions is presented. Furthermore, we propose to categorize the critical enabling techniques by explicit and implicit reasoning-based methods, and elaborate on how they evolve and contribute to modern content \& channel semantics-empowered communications. Besides reviewing and summarizing the latest efforts in SemCom, we discuss the relations with other communication levels (e.g., reliable and goal-oriented communications) from a holistic and unified viewpoint. Subsequently, in order to facilitate the future developments and industrial applications, we also highlight advanced practical techniques for boosting semantic accuracy, robustness, and large-scale scalability, just to mention a few. Finally, we discuss the technical challenges that shed light on future research opportunities.
translated by 谷歌翻译
Explainability of Graph Neural Networks (GNNs) is critical to various GNN applications but remains an open challenge. A convincing explanation should be both necessary and sufficient simultaneously. However, existing GNN explaining approaches focus on only one of the two aspects, necessity or sufficiency, or a trade-off between the two. To search for the most necessary and sufficient explanation, the Probability of Necessity and Sufficiency (PNS) can be applied since it can mathematically quantify the necessity and sufficiency of an explanation. Nevertheless, the difficulty of obtaining PNS due to non-monotonicity and the challenge of counterfactual estimation limits its wide use. To address the non-identifiability of PNS, we resort to a lower bound of PNS that can be optimized via counterfactual estimation, and propose Necessary and Sufficient Explanation for GNN (NSEG) via optimizing that lower bound. Specifically, we employ nearest neighbor matching to generate counterfactual samples for the features, which is different from the random perturbation. In particular, NSEG combines the edges and node features to generate an explanation, where the common edge explanation is a special case of the combined explanation. Empirical study shows that NSEG achieves excellent performance in generating the most necessary and sufficient explanations among a series of state-of-the-art methods.
translated by 谷歌翻译
Most Deep Learning (DL) based Compressed Sensing (DCS) algorithms adopt a single neural network for signal reconstruction, and fail to jointly consider the influences of the sampling operation for reconstruction. In this paper, we propose unified framework, which jointly considers the sampling and reconstruction process for image compressive sensing based on well-designed cascade neural networks. Two sub-networks, which are the sampling sub-network and the reconstruction sub-network, are included in the proposed framework. In the sampling sub-network, an adaptive full connected layer instead of the traditional random matrix is used to mimic the sampling operator. In the reconstruction sub-network, a cascade network combining stacked denoising autoencoder (SDA) and convolutional neural network (CNN) is designed to reconstruct signals. The SDA is used to solve the signal mapping problem and the signals are initially reconstructed. Furthermore, CNN is used to fully recover the structure and texture features of the image to obtain better reconstruction performance. Extensive experiments show that this framework outperforms many other state-of-the-art methods, especially at low sampling rates.
translated by 谷歌翻译
Multi-view graph clustering (MGC) methods are increasingly being studied due to the explosion of multi-view data with graph structural information. The critical point of MGC is to better utilize the view-specific and view-common information in features and graphs of multiple views. However, existing works have an inherent limitation that they are unable to concurrently utilize the consensus graph information across multiple graphs and the view-specific feature information. To address this issue, we propose Variational Graph Generator for Multi-View Graph Clustering (VGMGC). Specifically, a novel variational graph generator is proposed to extract common information among multiple graphs. This generator infers a reliable variational consensus graph based on a priori assumption over multiple graphs. Then a simple yet effective graph encoder in conjunction with the multi-view clustering objective is presented to learn the desired graph embeddings for clustering, which embeds the inferred view-common graph and view-specific graphs together with features. Finally, theoretical results illustrate the rationality of VGMGC by analyzing the uncertainty of the inferred consensus graph with information bottleneck principle. Extensive experiments demonstrate the superior performance of our VGMGC over SOTAs.
translated by 谷歌翻译
与传统CS方法相比,基于深度学习(DL)的压缩传感(CS)已被应用于图像重建的更好性能。但是,大多数现有的DL方法都利用逐个块测量,每个测量块分别恢复,这引入了重建的有害阻塞效应。此外,这些方法的神经元接受场被设计为每一层的大小相同,这只能收集单尺度的空间信息,并对重建过程产生负面影响。本文提出了一个新的框架,称为CS测量和重建的多尺度扩张卷积神经网络(MSDCNN)。在测量期间,我们直接从训练有素的测量网络中获得所有测量,该测量网络采用了完全卷积结构,并通过输入图像与重建网络共同训练。它不必将其切成块,从而有效地避免了块效应。在重建期间,我们提出了多尺度特征提取(MFE)体系结构,以模仿人类视觉系统以捕获同一功能映射的多尺度特征,从而增强了框架的图像特征提取能力并提高了框架的性能并提高了框架的性能。影像重建。在MFE中,有多个并行卷积通道以获取多尺度特征信息。然后,将多尺度功能信息融合在一起,并以高质量重建原始图像。我们的实验结果表明,根据PSNR和SSIM,该提出的方法对最新方法的性能有利。
translated by 谷歌翻译
信息指标的年龄无法正确描述状态更新的内在语义。在一个智能反映表面上的合作中继通信系统中,我们提出了语义年龄(AOS),用于测量状态更新的语义新鲜度。具体而言,我们专注于从源节点(SN)到目标的状态更新,该状态被称为马尔可夫决策过程(MDP)。 SN的目的是在最大发射功率约束下最大程度地提高AOS和能源消耗的预期满意度。为了寻求最佳的控制政策,我们首先在派利时间差异学习框架下推出了在线深层演员批评(DAC)学习方案。但是,实践实施在线DAC在SN和系统之间无限重复的互动中构成了关键的挑战,这可能是危险的,尤其是在探索过程中。然后,我们提出了一个新颖的离线DAC方案,该方案估算了先前收集的数据集的最佳控制策略,而无需与系统进行任何进一步的交互。数值实验验证了理论结果,并表明我们的离线DAC方案在平均效用方面显着优于在线DAC方案和最具代表性的基线,这表明了对数据集质量的强大鲁棒性。
translated by 谷歌翻译